Skip to content

3 develop testing guidelines - #5

Closed
emi728 wants to merge 9 commits into
masterfrom
3-develop-testing-guidelines
Closed

3 develop testing guidelines#5
emi728 wants to merge 9 commits into
masterfrom
3-develop-testing-guidelines

Conversation

@emi728

@emi728 emi728 commented May 13, 2026

Copy link
Copy Markdown
Collaborator

No description provided.

mitchell7970 and others added 9 commits May 4, 2026 13:45
- Generated with nf-core tools v4.0.2
- Includes CI workflows, linting config, base/test profiles
- Adds community boilerplate (LICENSE, CODE_OF_CONDUCT, CITATIONS)
- Includes nf-core shared subworkflows and modules
- column_standardise: map CSV headers to ABCD schema terms (WP6)
- biodiv_annotate: wrap BiodivPortal Annotator REST API (WP2)
- taxonomy_classify: resolve taxon names to GBIF/WFO IDs (WP3)
- outlier_detect: statistical + LLM quality detection (WP4)
- provisional_concepts: register unresolved terms (WP5)
- rdf_transform: tabular data to RDF triples (WP7)
- collect_reports: aggregate per-step quality reports

All modules are stub implementations ready for WP teams to replace.
- main.nf: entry point accepting --input CSV and --mapping_schema
- workflows/biodivpipeline.nf: orchestrates all 7 modules
  - Step 1: Column standardisation (WP6)
  - Step 2: Parallel annotation (WP2) + taxonomy (WP3) + outlier detection (WP4)
  - Step 3: Provisional concept registration (WP5, after WP2)
  - Step 4: RDF transformation (WP7, combines all upstream outputs)
  - Step 5: Report collection
- nextflow.config: add input/mapping_schema params
- nextflow_schema.json: update input parameter validation for CSV data
- Remove FASTQC/MultiQC module references from workflow
- Replace samplesheet parsing with direct CSV file channel
- Remove genome/igenomes/fasta parameter references
- Update PIPELINE_INITIALISATION to validate CSV input
- Replace validateInputSamplesheet with validateInputParameters
- Update modules.config: per-module publishDir for our 7 modules
- Update test.config: point to test_data/sample.csv
- test_data/sample.csv: 109 records sampled from BELEGE_aus_D (every 1000th row)
- modules/stubs/: FastAPI mock service for end-to-end testing
  - Dockerfile + stub_service.py (POST /process, GET /health)
- assets/default_mapping.jsonld: stub DwC RDF mapping schema
  - Maps BGBM CSV columns to Darwin Core terms

Pipeline runs end-to-end with: nextflow run main.nf -profile test
Replace nf-core genomics boilerplate with project-specific content:
- README.md: project summary, Mermaid diagrams, quick start, output tree
- docs/usage.md: CSV input format, parameters, sequence diagram
- docs/output.md: all 18 output files across 7 directories
- docs/CONTRIBUTING.md: branch model, conventional commits, module interface
- docs/README.md: documentation index with tech stack table

Remove unused template files:
- modules/nf-core/fastqc, modules/nf-core/multiqc (genomics QC)
- assets: samplesheet.csv, schema_input.json, multiqc_config.yml, logos
- conf: igenomes configs, container lock files, test_full.config
- tower.yml, methods_description_template.yml

Clean code references:
- nextflow.config: remove igenomes include, test_full profile
- modules.json: remove fastqc/multiqc entries
- main.nf, subworkflow: remove multiqc_report channel
- subworkflow: remove samplesheetToList import, FastQC/MultiQC citations
WP1 owns the integration backbone, not module internals. Strip stub scripts
to minimal valid-output emitters and remove pre-claimed domain content:

- Delete modules/stubs/ (FastAPI HTTP service contradicts Pattern A)
- Replace 6 module main.nf stubs with minimal emitters
- Strip default_mapping.jsonld to empty placeholder
- Replace biodiv/wpX-*:latest container refs with placeholder image
- Remove WP-to-module assignments from workflow comments
- Drop BGBM example from input description

Module teams (WP2-WP7) own real implementations, container images, and
domain content. WP1 ships interface contracts.
- README.md: remove module diagram, dataset claims, project-structure
  module descriptions, credits section
- docs/output.md: reduce to directory layout + Nextflow pipeline_info
- docs/README.md: shrink to TOC + tech stack
- docs/usage.md: remove dataset specifics, column listing, unzip step,
  sequence diagram
- docs/CONTRIBUTING.md: remove project contacts, dataset record count
- nextflow.config: empty manifest.contributors placeholder
- .nf-core.yml: clear author placeholder
- CITATIONS.md: drop tools not used in this pipeline
@emi728 emi728 linked an issue May 13, 2026 that may be closed by this pull request
@mitchell7970
mitchell7970 deleted the 3-develop-testing-guidelines branch May 15, 2026 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Develop Testing Guidelines

3 participants